<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">



<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />

        <title>
            
    
                The Unicode Chapter
             &mdash;
    Mako 1.3.2 Documentation

        </title>

        
            <!-- begin iterate through site-imported + sphinx environment css_files -->
                <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
                <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
                <link rel="stylesheet" href="_static/default.css" type="text/css" />
                <link rel="stylesheet" href="_static/changelog.css" type="text/css" />
                <link rel="stylesheet" href="_static/sphinx_paramlinks.css" type="text/css" />
                <link rel="stylesheet" href="_static/docs.css" type="text/css" />
            <!-- end iterate through site-imported + sphinx environment css_files -->
        

        
    

    <!-- begin layout.mako headers -->

    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="top" title="Mako 1.3.2 Documentation" href="index.html" />
        <link rel="next" title="Caching" href="caching.html" />
        <link rel="prev" title="Filtering and Buffering" href="filtering.html" />
    <!-- end layout.mako headers -->


    </head>
    <body>
        










<div id="docs-container">



<div id="docs-header">
    <h1>Mako 1.3.2 Documentation</h1>

    <div id="docs-search">
    Search:
    <form class="search" action="search.html" method="get">
      <input type="text" name="q" size="18" /> <input type="submit" value="Search" />
      <input type="hidden" name="check_keywords" value="yes" />
      <input type="hidden" name="area" value="default" />
    </form>
    </div>

    <div id="docs-version-header">
        Release: <span class="version-num">1.3.2</span>

    </div>

</div>

<div id="docs-top-navigation">
    <div id="docs-top-page-control" class="docs-navigation-links">
        <ul>
            <li>Prev:
            <a href="filtering.html" title="previous chapter">Filtering and Buffering</a>
            </li>
            <li>Next:
            <a href="caching.html" title="next chapter">Caching</a>
            </li>

        <li>
            <a href="index.html">Table of Contents</a> |
            <a href="genindex.html">Index</a>
        </li>
        </ul>
    </div>

    <div id="docs-navigation-banner">
        <a href="index.html">Mako 1.3.2 Documentation</a>
        » 
                The Unicode Chapter
            

        <h2>
            
                The Unicode Chapter
            
        </h2>
    </div>

</div>

<div id="docs-body-container">


    <div id="docs-sidebar">
    <div id="sidebar-banner">
        
    </div>

    <h3><a href="index.html">Table of Contents</a></h3>
    <div id="sidebar-toc"><ul>
<li><a class="reference internal" href="#">The Unicode Chapter</a><ul>
<li><a class="reference internal" href="#specifying-the-encoding-of-a-template-file">Specifying the Encoding of a Template File</a></li>
<li><a class="reference internal" href="#handling-expressions">Handling Expressions</a></li>
<li><a class="reference internal" href="#defining-output-encoding">Defining Output Encoding</a></li>
</ul>
</li>
</ul>
</div>

    <h4>Previous Topic</h4>
    <p>
    <a href="filtering.html" title="previous chapter">Filtering and Buffering</a>
    </p>
    <h4>Next Topic</h4>
    <p>
    <a href="caching.html" title="next chapter">Caching</a>
    </p>

    <h4>Quick Search</h4>
    <p>
    <form class="search" action="search.html" method="get">
      <input type="text" name="q" size="18" /> <input type="submit" value="Search" />
      <input type="hidden" name="check_keywords" value="yes" />
      <input type="hidden" name="area" value="default" />
    </form>
    </p>

    </div>

    <div id="docs-body" class="withsidebar" >
        
<section id="the-unicode-chapter">
<span id="unicode-toplevel"></span><h1>The Unicode Chapter<a class="headerlink" href="#the-unicode-chapter" title="Link to this heading">¶</a></h1>
<p>In normal Mako operation, all parsed template constructs and
output streams are handled internally as Python 3 <code class="docutils literal notranslate"><span class="pre">str</span></code> (Unicode)
objects. It’s only at the point of <a class="reference internal" href="usage.html#mako.template.Template.render" title="mako.template.Template.render"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Template.render()</span></code></a> that this stream of Unicode objects may be rendered into whatever the desired output encoding
is. The implication here is that the template developer must
:ensure that <a class="reference internal" href="#set-template-file-encoding"><span class="std std-ref">the encoding of all non-ASCII templates is explicit</span></a> (still required in Python 3, although Mako defaults to <code class="docutils literal notranslate"><span class="pre">utf-8</span></code>),
that <a class="reference internal" href="#handling-non-ascii-expressions"><span class="std std-ref">all non-ASCII-encoded expressions are in one way or another
converted to unicode</span></a>
(not much of a burden in Python 3), and that <a class="reference internal" href="#defining-output-encoding"><span class="std std-ref">the output stream of the
template is handled as a unicode stream being encoded to some
encoding</span></a> (still required in Python 3).</p>
<section id="specifying-the-encoding-of-a-template-file">
<span id="set-template-file-encoding"></span><h2>Specifying the Encoding of a Template File<a class="headerlink" href="#specifying-the-encoding-of-a-template-file" title="Link to this heading">¶</a></h2>
<div class="versionchanged">
<p><span class="versionmodified changed">Changed in version 1.1.3: </span>As of Mako 1.1.3, the default template encoding is “utf-8”.  Previously, a
Python “magic encoding comment” was required for templates that were not
using ASCII.</p>
</div>
<p>Mako templates support Python’s “magic encoding comment” syntax
described in  <a class="reference external" href="http://www.python.org/dev/peps/pep-0263/">pep-0263</a>:</p>
<div class="highlight-mako notranslate"><div class="highlight"><pre><span></span><span class="c1">## -*- coding: utf-8 -*-</span>

<span class="x">Alors vous imaginez ma surprise, au lever du jour, quand</span>
<span class="x">une drôle de petite voix m’a réveillé. Elle disait:</span>
<span class="x"> « S’il vous plaît… dessine-moi un mouton! »</span></pre></div>
</div>
<p>As an alternative, the template encoding can be specified
programmatically to either <a class="reference internal" href="usage.html#mako.template.Template" title="mako.template.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">Template</span></code></a> or <a class="reference internal" href="usage.html#mako.lookup.TemplateLookup" title="mako.lookup.TemplateLookup"><code class="xref py py-class docutils literal notranslate"><span class="pre">TemplateLookup</span></code></a> via
the <code class="docutils literal notranslate"><span class="pre">input_encoding</span></code> parameter:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">t</span> <span class="o">=</span> <span class="n">TemplateLookup</span><span class="p">(</span><span class="n">directories</span><span class="o">=</span><span class="p">[</span><span class="s1">&#39;./&#39;</span><span class="p">],</span> <span class="n">input_encoding</span><span class="o">=</span><span class="s1">&#39;utf-8&#39;</span><span class="p">)</span></pre></div>
</div>
<p>The above will assume all located templates specify <code class="docutils literal notranslate"><span class="pre">utf-8</span></code>
encoding, unless the template itself contains its own magic
encoding comment, which takes precedence.</p>
</section>
<section id="handling-expressions">
<span id="handling-non-ascii-expressions"></span><h2>Handling Expressions<a class="headerlink" href="#handling-expressions" title="Link to this heading">¶</a></h2>
<p>The next area that encoding comes into play is in expression
constructs. By default, Mako’s treatment of an expression like
this:</p>
<div class="highlight-mako notranslate"><div class="highlight"><pre><span></span><span class="cp">${</span><span class="s2">&quot;hello world&quot;</span><span class="cp">}</span></pre></div>
</div>
<p>looks something like this:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">context</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="s2">&quot;hello world&quot;</span><span class="p">))</span></pre></div>
</div>
<p>That is, <strong>the output of all expressions is run through the
``str`` built-in</strong>. This is the default setting, and can be
modified to expect various encodings. The <code class="docutils literal notranslate"><span class="pre">str</span></code> step serves
both the purpose of rendering non-string expressions into
strings (such as integers or objects which contain <code class="docutils literal notranslate"><span class="pre">__str()__</span></code>
methods), and to ensure that the final output stream is
constructed as a Unicode object. The main implication of this is
that <strong>any raw byte-strings that contain an encoding other than
ASCII must first be decoded to a Python unicode object</strong>.</p>
<p>Similarly, if you are reading data from a file that is streaming
bytes, or returning data from some object that is returning a
Python byte-string containing a non-ASCII encoding, you have to
explicitly decode to Unicode first, such as:</p>
<div class="highlight-mako notranslate"><div class="highlight"><pre><span></span><span class="cp">${</span><span class="n">call_my_object</span><span class="p">()</span><span class="o">.</span><span class="n">decode</span><span class="p">(</span><span class="s1">&#39;utf-8&#39;</span><span class="p">)</span><span class="cp">}</span></pre></div>
</div>
<p>Note that filehandles acquired by <code class="docutils literal notranslate"><span class="pre">open()</span></code> in Python 3 default
to returning “text”: that is, the decoding is done for you. See
Python 3’s documentation for the <code class="docutils literal notranslate"><span class="pre">open()</span></code> built-in for details on
this.</p>
<p>If you want a certain encoding applied to <em>all</em> expressions,
override the <code class="docutils literal notranslate"><span class="pre">str</span></code> builtin with the <code class="docutils literal notranslate"><span class="pre">decode</span></code> built-in at the
<a class="reference internal" href="usage.html#mako.template.Template" title="mako.template.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">Template</span></code></a> or <a class="reference internal" href="usage.html#mako.lookup.TemplateLookup" title="mako.lookup.TemplateLookup"><code class="xref py py-class docutils literal notranslate"><span class="pre">TemplateLookup</span></code></a> level:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">t</span> <span class="o">=</span> <span class="n">Template</span><span class="p">(</span><span class="n">templatetext</span><span class="p">,</span> <span class="n">default_filters</span><span class="o">=</span><span class="p">[</span><span class="s1">&#39;decode.utf8&#39;</span><span class="p">])</span></pre></div>
</div>
<p>Note that the built-in <code class="docutils literal notranslate"><span class="pre">decode</span></code> object is slower than the
<code class="docutils literal notranslate"><span class="pre">str</span></code> function, since unlike <code class="docutils literal notranslate"><span class="pre">str</span></code> it’s not a Python
built-in, and it also checks the type of the incoming data to
determine if string conversion is needed first.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">default_filters</span></code> argument can be used to entirely customize
the filtering process of expressions. This argument is described
in <a class="reference internal" href="filtering.html#filtering-default-filters"><span class="std std-ref">The default_filters Argument</span></a>.</p>
</section>
<section id="defining-output-encoding">
<span id="id1"></span><h2>Defining Output Encoding<a class="headerlink" href="#defining-output-encoding" title="Link to this heading">¶</a></h2>
<p>Now that we have a template which produces a pure Unicode output
stream, all the hard work is done. We can take the output and do
anything with it.</p>
<p>As stated in the <a class="reference internal" href="usage.html"><span class="doc">“Usage” chapter</span></a>, both <a class="reference internal" href="usage.html#mako.template.Template" title="mako.template.Template"><code class="xref py py-class docutils literal notranslate"><span class="pre">Template</span></code></a> and
<a class="reference internal" href="usage.html#mako.lookup.TemplateLookup" title="mako.lookup.TemplateLookup"><code class="xref py py-class docutils literal notranslate"><span class="pre">TemplateLookup</span></code></a> accept <code class="docutils literal notranslate"><span class="pre">output_encoding</span></code> and <code class="docutils literal notranslate"><span class="pre">encoding_errors</span></code>
parameters which can be used to encode the output in any Python
supported codec:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">mako.template</span> <span class="kn">import</span> <span class="n">Template</span>
<span class="kn">from</span> <span class="nn">mako.lookup</span> <span class="kn">import</span> <span class="n">TemplateLookup</span>

<span class="n">mylookup</span> <span class="o">=</span> <span class="n">TemplateLookup</span><span class="p">(</span><span class="n">directories</span><span class="o">=</span><span class="p">[</span><span class="s1">&#39;/docs&#39;</span><span class="p">],</span> <span class="n">output_encoding</span><span class="o">=</span><span class="s1">&#39;utf-8&#39;</span><span class="p">,</span> <span class="n">encoding_errors</span><span class="o">=</span><span class="s1">&#39;replace&#39;</span><span class="p">)</span>

<span class="n">mytemplate</span> <span class="o">=</span> <span class="n">mylookup</span><span class="o">.</span><span class="n">get_template</span><span class="p">(</span><span class="s2">&quot;foo.txt&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">mytemplate</span><span class="o">.</span><span class="n">render</span><span class="p">())</span></pre></div>
</div>
<p><a class="reference internal" href="usage.html#mako.template.Template.render" title="mako.template.Template.render"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Template.render()</span></code></a> will return a <code class="docutils literal notranslate"><span class="pre">bytes</span></code> object in Python 3 if an output
encoding is specified. By default it performs no encoding and
returns a native string.</p>
<p><a class="reference internal" href="usage.html#mako.template.Template.render_unicode" title="mako.template.Template.render_unicode"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Template.render_unicode()</span></code></a> will return the template output as a Python
<code class="docutils literal notranslate"><span class="pre">str</span></code> object:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="n">mytemplate</span><span class="o">.</span><span class="n">render_unicode</span><span class="p">())</span></pre></div>
</div>
<p>The above method disgards the output encoding keyword argument;
you can encode yourself by saying:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="n">mytemplate</span><span class="o">.</span><span class="n">render_unicode</span><span class="p">()</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">&#39;utf-8&#39;</span><span class="p">,</span> <span class="s1">&#39;replace&#39;</span><span class="p">))</span></pre></div>
</div>
</section>
</section>

    </div>

</div>

<div id="docs-bottom-navigation" class="docs-navigation-links">
        Previous:
        <a href="filtering.html" title="previous chapter">Filtering and Buffering</a>
        Next:
        <a href="caching.html" title="next chapter">Caching</a>

    <div id="docs-copyright">
        &copy; Copyright the Mako authors and contributors.
        Documentation generated using <a href="http://sphinx.pocoo.org/">Sphinx</a> 7.2.6
        with Mako templates.
    </div>
</div>

</div>



        
        

    <script type="text/javascript">
      var DOCUMENTATION_OPTIONS = {
          URL_ROOT:    './',
          VERSION:     '1.3.2',
          COLLAPSE_MODINDEX: false,
          FILE_SUFFIX: '.html'
      };
    </script>

    <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>

    <!-- begin iterate through sphinx environment script_files -->
        <script type="text/javascript" src="_static/jquery.js"></script>
        <script type="text/javascript" src="_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script type="text/javascript" src="_static/documentation_options.js"></script>
        <script type="text/javascript" src="_static/doctools.js"></script>
        <script type="text/javascript" src="_static/sphinx_highlight.js"></script>
    <!-- end iterate through sphinx environment script_files -->

    <script type="text/javascript" src="_static/init.js"></script>


    </body>
</html>


